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Virtual  or  Physical  Prototypes? 

Development  and  Testing  of  a  Prototyping  Planning  Tool 


Abstract 

A  new  prototyping  planning  tool  guides  designers  in  choosing  between  virtual  vs.  physical 
prototyping  strategies  based  on  answers  to  Likert-scale  questions.  We  developed  this  tool  to 
augment  prior  work  in  design  methods  seeking  to  facilitate  prototyping  strategy  development. 
This  new  tool  was  tested  in  a  pilot  experiment  with  engineering  students  individually  tasked 
with  optimizing  the  design  of  a  four-bar  linkage. 

All  students  were  given  a  design  problem  that  involved  designing  a  four-bar  linkage  to  be  used  to 
draw  a  specific  shape.  The  students  were  then  instructed  to  use  the  new  prototyping  planning 
tool  to  decide  whether  to  create  a  virtual  or  physical  prototype  of  a  four-bar  linkage,  with  the 
goal  of  maximizing  the  performance  metric  detailed  in  the  design  problem  statement.  This  paper 
describes  the  new  prototype  strategy  planning  tool,  the  pilot  experiment,  and  results  and 
conclusions.  The  very  encouraging  pilot  results  provide  a  template  and  strong  motivation  for 
conducting  a  larger  scale  experiment  with  a  sample  size  leading  to  statistically  significant 
findings. 

1  Introduction 

An  engineering  gmtoQge  (physical  or  virtual)  is  an  initial  manifestation  of  a  design  concept, 
either  a  scale  or  full-size  model  of  a  structure  or  piece  of  equipment,  which  can  be  used  to 
evaluate  performance,  form,  and/or  fit.  Prototyping  is  the  process  of  generating  prototype(s), 
usually  between  concept  generation  and  design  verification  stages.  Prototypes  provide  design 
engineers  the  opportunity  to:  determine  if  a  concept  is  technically  feasible,  optimize 
performance,  understand  interfaces  between  subsystems,  and/or  identify  potential  assembly  and 
manufacturing  issues,  in  addition,  prototypes  serve  as  an  effective  method  of  communicating  the 
functionality  and/or  progress  of  a  design  concept,  to  both  technical  and  non-technical  audiences, 
for  these  reasons  prototyping  is  an  integral  part  of  the  product  development  process. 

Prototypes  may  be  physical  or  virtual.  A  physical  prototype  is  the  preliminary  embodiment  of  a 
design  concept  in  a  tangible  model.  Physical  prototypes  may  be  fully  or  partially  functional,  and 
allow  for  sensory  evaluation  of  the  concept,  possibly  including  aesthetics  and  ergonomics,  in 
contrast,  virtual  prototypes  are  digital  mock-ups  (computer  simulations  and/or  analytical  models) 
of  physical  products  that  can  be  analyzed,  tested,  and  presented  in  order  to  serve  the  principal 
purposes  of  prototyping  in  the  product  development  process.  Computational  advances  have 
vastly  expanded  the  possibilities  of  virtual  prototyping  in  the  past  few  decades.  Practical 
examples  of  virtual  prototyping  techniques  include  3D  CAD  models  with  motion  analysis,  finite 
element  analysis,  manufacturability  evaluations,  and/or  computational  fluid  dynamics  (CFD.)  A 
lack  of  tangible  interaction  and  evaluation  distinguishes  virtual  prototypes  from  physical 
prototypes  .  Both  virtual  and  physical  prototypes  may  be  developed  for  an  entire  system  or  a 
specific  subsystem. 


2  Background 

Prototyping  is  a  vital  component  in  the  product  design  process  as  a  whole.  A  prototyping 
strategy  refers  here  to  the  set  of  choices  guiding  development  of prototype  (s)1 2 .  A  general 
prototyping  strategy  (such  as  “one  should  prototype  multiple  concepts  early”)  leads  to  a  project- 
specific  prototyping  strategy  (such  as  “prototype  concepts  A,  D,  and  E  by  week  #3.”)  Most 
currently  published  structured  prototyping  approaches  (strategies)  focus  on  management  logistics 
aspects  such  as  lead  times,  budgets,  and  project  efficiency3.  However,  Otto  and  Wood4  provide  a 
foundation  for  an  engineering  approach  to  prototyping  strategy  in  the  form  of  a  basic  method  for 
designing  physical  prototypes  and  guidelines  for  prototype  development.  Additionally,  Otto  and 
Wood  cover  analytical  (virtual)  and  physical  prototyping  techniques  and  appropriate  testing 
procedures  to  ensure  that  physical  models  satisfy  design  requirements.  The  authors  acknowledge 
that  virtual  modelling  is  important  in  the  prototyping  process,  but  they  recommend  that  designers 
must  ultimately  develop  and  test  physical  prototypes  for  the  successful  instantiation  of  design 
concepts. 

Recently  a  diverse  research  team  presented  work  towards  generalized  methodologies  for 
developing  project-specific  prototyping  strategies5.  This  methodology  simply  translates  the 
context  of  a  specific  design  problem  into  prototyping  decisions,  yielding  a  project-specific 
prototyping  strategy.  In  other  words,  the  prototyping  strategy  formation  methodology  uses  the 
independent  variables  of  a  design  problem  (e.g.  available  budget/time,  difficulty  in  meeting 
design  requirements,  and  designer’s  experience)  in  order  to  derive  dependent  prototyping 
strategy  variables  (e.g.  number  of  prototypes  to  build,  prototype  scaling,  and  subsystem 
isolation).  I  hese  dependent  strategy  variables,  representing  many  critical  prototyping  decisions, 
were  amalgamated  from  heuristics  for  prototyping  best  practices  outlined  by  Moe6,  Christie7,  and 
Viswanathan  .  This  prototyping  strategy  formation  method  provides  a  systematic  framework  to 
translate  independent  context  variables  into  dependent  prototyping  strategy  variables  in  the 
following  four  steps  (Figure  1): 

1.  Predict  how  many  iterations  each  concept  requires  to  satisfy  design  requirements  (in  light 
of.  designer  s  experience,  design  requirement  difficulty,  and  design  requirement  rigidity.) 

2.  Determine  appropriate  prototype  scaling,  subsystem  isolation,  and  functional  relaxation 
for  each  iteration  of  each  concept  (diagrammatic  flowcharts  aid  this  step.) 

3.  Determine  which  concepts  to  prototype  in  parallel,  based  on  available  budget  and  time. 

4.  Document  the  resulting  prototyping  strategy. 
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Figure  1:  Flowchart  of  Prototyping  Strategy  Method5 

The  above  methodology  was  previously  experimentally  evaluated  in  a  controlled  design 
environment  using  primarily  mechanical  engineering  students5.  The  published  experiment 
results  indicate  using  the  prototyping  strategy  formation  method  above  is  positively  correlated 
with  early-stage  design  success.  Thus,  implementing  this  method  can  potentially  improve  design 
performance  while  increasing  the  likelihood  of  staying  within  budget  and  time  constraints.  The 
need  for  more  comprehensive  prototyping  planning  and  this  ground-breaking  research  on  an 
engineering  approach  to  prototyping  strategy  formation  both  motivate  and  shape  the  new  work 
presented  in  this  paper. 

This  paper  supplements  the  above  prototyping  strategy  formation  method  pioneered  by  Cambum 
et  al.  by  adding  a  new  prototyping  decision  -  whether  a  prototype  will  be  virtual  or  physical. 

The  goal  of  this  study  is  a  systematic  decision  tool  guiding  the  choice  between  virtual  or  physical 
prototyping  for  generalized  design  problems. 

No  research  was  located  addressing  a  structured  method  aiding  engineering  designers  in  deciding 
between  virtual  and  physical  prototyping.  However,  Ulrich  and  EppingeP  detail  the  usefulness 
of  considering  virtual  versus  physical  prototyping  in  a  generalized  description  of  the  prototyping 
process.  By  charting  prototyping  decisions  in  two  dimensional  space  (Figure  2),  they  provide  a 
graphical  decision  making  tool  based  on  the  relative  accuracy  and  expense  of  virtual  versus 
physical  prototypes. 
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Figure  2:  Decision  Trade-off  between  Virtual  and  Physical  Models  (Adapted  from  Ulrich  and  Eppinger9) 

Ulrich  and  Eppingers  only  caveat  is  that  comprehensive  virtual  prototypes  are  generally  not 
feasible.  However,  there  are  some  instances  when  physical  prototypes  are  prohibitively 
expensive.  For  example,  Northrop  Grumman10  has  virtually  modeled  an  entire  aircraft  carrier 
with  over  three  million  parts  (Figure  3.)  This  complex  virtual  model  allows  engineers  to  foresee 
potential  piping  layout  issues,  predict  overall  buoyancy/draft  height/center  of  mass,  and  estimate 
drag  forces  without  constructing  a  costly  physical  model. 


Figure  3 :  Virtual  Model  of  Aircraft  Carrier  (image  courtesy  Northrup  Grumman)10 

3  Methodology 

Choosing  between  a  virtual  or  physical  prototype  is  a  critical  decision  in  the  product  design 
process.  Selection  of  the  type  of  prototype(s)  will  most  likely  be  determined  based  on  budget 
and  time  constraints  as  well  as  the  experience  of  a  design  team.  A  structured  prototyping 
strategy  lormatton  method  addressing  the  choice  of  virtual  or  physical  prototypes  will  be  a  useful 
addition  to  the  prototyping  strategy  formation  tool  developed  by  Cambum  et  al.5  described 
above. 


I  here  foie,  a  decision  making  tool  lor  choosing  between  virtual  and  physical  prototypes  was 
developed  to  build  upon  the  existing  prototyping  strategy  formation  tool.  A  major  contribution 
of  this  work  is  the  use  of  a  heuristics-based  approach,  rather  than  the  strictly  quantitative 
approach  of  the  prior  work.  The  new  virtual-vs-physical  module  of  the  prototyping  strategy 
formation  method  includes  a  newly  developed  tool  employing  Likert-scale  questions  (Appendix 
A.) 

The  pilot  experimental  study  reported  here  tests  this  heuristics-based,  Likert-scale  tool.  A 
classical  four-bar  linkage  design  problem  was  chosen  for  a  controlled  experiment  based  on 
practical  considerations.  The  feasibility  of  prototyping  four-bar  linkages  both  physically  and 
virtually,  with  basic  materials  and  easy-to-use  software,  enables  testing  of  the  new  virtual  vs. 
physical  module  of  the  prototyping  strategy  formation  method. 

All  eight  participants  in  this  pilot  experiment  were  junior  or  senior  mechanical  engineering 
students,  with  at  least  basic  familiarity  with  tour-bar  linkage  design.  Each  participant  worked 
individually.  As  shown  in  the  experiment  worksheet  in  Appendix  A,  the  experiment  began  with 
a  five  minute  introduction.  During  this  time  the  difference  between  virtual  and  physical 
prototypes  was  defined.  In  addition,  participants  were  shown  both  a  graphical  depiction  and 
physical  example  of  a  four-bar  linkage.  Next,  Grashof  s  Law  was  presented  in  order  to  inform 
participants  of  the  condition  necessary  to  achieve  continual  rotation  of  the  shortest  link  in  a  four- 
bar  linkage,  i.e.,  the  sum  of  the  shortest  and  longest  links  must  be  less  than  the  sum  of  the  other 
two  links. 

Participants  were  then  instructed  to  complete  an  initial  Likert-scale  survey  (Appendix  A),  and 
record  their  familiarity  with  four-bar  linkages,  experience  using  computer  simulation  software, 
experience  building  physical  models,  and  preference  of  using  software  versus  building  physical 
models.  Next,  creating  a  virtual  four-bar  linkage  using  GIM11,12  software  was  briefly  detailed. 
The  free  (with  registration)  GIM  software  provides  an  easy  interactive  environment  for  the 
design  and  simulation  of  simple  linkages  (Figure  4.) 


After  a  brief  demonstration  of  the  GIM  software,  participants  were  presented  with  the  following 
materials  and  tools  (Figure  5)  to  potentially  construct  a  physical  four-bar  linkage: 

•  Four  precut  foam  board  pieces 

•  Four  detachable  pins 

•  Hole-punch  (for  installing  detachable  pins) 

•  Scissors  (for  cutting  foam  board  to  the  desired  length) 

•  Pencil  and  paper  (for  marking  the  trajectory  of  a  four-bar  linkage) 


Figure  5:  Materials  for  Four-bar  Linkage  Physical  Modelling,  Excluding  Paper 

F  articipants  were  then  presented  with  the  problem  of  designing  a  continuously  rotating  four- bar 
linkage  to  draw  the  longest  possible  horizontal  shape  (the  closest  approximation  of  a  straight 
line).  The  design  objective  is  to  maximize  AX  /  AY  (Figure  6.)  Participants  were  permitted  an 
unl  imited  number  of  modifications  within  a  30  minute  time  limit. 


F  igure  6;  Depiction  ot  Performance  Metric:  Maximize  AX  /  AY 

After  participants  had  an  understanding  of  the  design  problem  and  the  process  for  creating  both 
virtual  and  physical  tour-bar  linkage  prototypes,  they  were  instructed  to  complete  a  second 


Likert-scale  survey  (Figure  7.)  Based  on  the  sum  of  their  survey  responses  (bottom  of  Figure  7), 
participants  then  chose  to  either  virtually  or  physically  prototype  a  four-bar  linkage  in  order  to 
achieve  the  design  objective. 
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Figure  7:  Likert-scale  Survey  for  Informing  Choice  between  Virtual  and  Physical  Prototyping 

The  Likert-scale  in  Figure  7  expands  upon  Ulrich  and  Eppinger’s  two-axis  graph  of  suggested 
choices  based  on  the  relative  accuracy  of  virtual  with  respect  to  physical  models  versus  the 
relative  effort  of  virtual  with  respect  to  physical  models.  Question  (a)  addresses  participants’ 
perception  of  the  ratio  of  accuracy  between  virtual  and  physical  models,  and  question  (b) 
addresses  the  ratio  of  effort  between  virtual  and  physical  models.  Question  (c)  has  the  designer 
consider  the  number  of  design  iterations  to  address  the  relative  ratio  of  both  effort  and  time  of 
virtual  compared  to  physical  models.  Participants  use  the  sum  of  their  responses  to  choose 
which  type  of  prototype  to  create. 

Participants  who  chose  virtual  prototyping  received  a  short  (<5  min.)  GIM  software  tutorial, 
while  those  choosing  physical  prototyping  received  a  brief  (<5  min.)  demonstration  of  physical 
construction  with  the  provided  materials.  Each  participant  recorded  their  prototyping  start  and 
end  times  with  a  maximum  of  30  minutes  allowed. 

Finally,  after  completion  of  either  a  virtual  or  physical  prototype,  all  participants  completed  an 
exit  survey.  The  exit  survey  recorded  participants’  opinion  of  the  choices  they  made  and  of  the 
Likert-scale  as  a  decision  making  tool. 

4  Results 

Two  out  of  the  eight  participants  in  this  pilot  study  chose  to  create  a  physical  prototype  of  their 
four-bar  linkage,  and  Figure  8  pictures  an  example  being  used  to  draw  a  pencil  line  on  paper. 


Figure  8:  Example  of  Four-bar  Linkage  Physical  Prototype,  with  Pencil  to  Draw  a  Line  on  Paper 

Participants  choosing  virtual  prototyping  achieved  AX  /  AY  ratios  on  average  six  times  higher 
than  participants  using  physical  prototypes.  One  additional  participant  not  included  here  was 
assigned  (rather  than  given  a  choice)  to  physically  prototype,  and  outperformed  the  physical 
prototypers  in  this  study  by  a  factor  of  3  (achieving  a  AX  /  AY  =  9.5,  although  still  less  than  the 
21.6  average  of  the  six  virtual  prototypes.)  This  exploratory  pilot  experiment  with  a  small 
sample  size  is  intended  to  guide  design  of  a  larger  experiment  with  more  statistically  robust 
results. 

I  able  1  presents  a  summary  of  the  data  obtained  in  this  pilot  experimental  study,  including  the 
AX  /  AY  ratio  performance  metrics  and  time  to  complete  prototyping.  Appendix  B  presents 
more  detailed  data. 


Table  1:  Summary  of  Experiment  Data 


Average: 

Virtual 

Average: 

Physical 

Delta 

(V-P) 

initial  Survey 

1  have  an  understanding  of  four-bar  linkages 

0.8 

1.0 

-0.2 

1  have  experience  using  software 

1,3 

0.5 

0.8 

1  prefer  to  design  using  software 

0.7 

-1.0 

1.7 

1  have  experience  building  physical  models 

0,7 

1,5 

-0.8 

Likert  Scale 

VP  will  require  less  time  than  PP 

0.7 

0.0 

0.7 

VP  will  be  sufficiently  accurate 

1.3 

0.0 

1.3 

Prototyping  a  four-bar  linkage  will  require  many 

iterations 

1.2 

-10 

2,2 

Exit  Survey 

VP  is  the  best  technique  for  designing  four- bars 

1.7 

0.0 

1.7 

GIM  is  a  useful  tool  for  VP  four-bars 

1.7 

0.5 

12 

Likert  Scale  was  useful  in  choosing  VP  or  PP 

0.7 

0.0 

0.7 

1  will  consider  using  VP  in  future  designs 

1.7 

1.5 

0.2 

1  chose  the  best  technique  for  my  prototype 

1-7 

-1.5 

3.2 

Time  to  Complete  (minutes) 

21 

26 

-5 

Performance  Ratio  (AX  /  AY) 

21.6 

3.5 

18.2 

5  Conclusions  and  Future  Work 

this  pilot  study,  itself  a  prototype  for  future  experiments,  provides  insight  into  development  of  a 
tool  aiding  designer  choice  between  virtual  and  physical  prototypes.  This  paper  presents  a 
Likert-scale  guide  to  choosing  virtual  vs.  physical  prototyping,  which  can  be  a  useful  addition  to 
a  larger  prototyping  strategy  formulation  method  such  as  that  proposed  by  Cambum  et  al5.  It  is 
not  the  intention  of  this  small  sample  size  pilot  to  make  statistical  claims,  but  rather  to 
demonstrate  the  viability  of  the  experiment  and  provide  a  foundation  and  compelling  motivation 
to  conduct  it  on  a  much  larger  scale. 

Future  work  will  improve  the  experiment  detailed  in  this  paper  and  obtain  results  for  a  larger 
sample  size.  In  addition,  testing  this  method  with  new  design  problems,  in  which  the  choice 
between  virtual  and  physical  models  is  not  obvious,  will  provide  more  generalizable  results* 
Potential  design  problems  must  use  simple  and  readily- available  computer  software  for  practical 
reasons. 


Additional  research  will  seek  deeper  understand  of  what  designers  learn  from  tactile  engagement 
while  building  physical  prototypes  (such  as  fit  and  form),  in  contrast  to  the  virtual  experience  of 


software  manipulations.  Incorporating  the  importance  of  human-prototype  interaction  as  a 
heuristic  in  decision  making  may  enhance  prototyping  strategies. 

Finally,  investigating  any  potential  correlations  between  MBTI  types  of  participants  and  virtual 
vs.  physical  preference  and  performance  outcomes  might  lead  to  the  development  of  additional 
heuristics. 
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Appendix  A:  Experiment  Worksheet 


Name: 


Date: 


Four-Bar  Linkages 

Virtual  or  Physical  Prototyping  Experiment 

1.  Introduction  to  four-bar  linkages: 

•  Grashof  s  Law*,  {shortest  link  +  longest  link)  <  (sum  of  remaining  i  links) 

•  Virtual  Prototype  -  a  computer  simulation  (CAD  model,  motion  analysis,  FEA,  CFD,  etc.)  of  a  product  that 
can  be  analyzed,  tested,  and  modified. 

•  Physical  Prototype  -  a  tangible,  physical  model  of  a  product  that  can  be  analyzed,  tested,  and  modified. 


R  =  rocker  link 
G  =  ground  link 
F  =  follower  link 
C  =  coupler  link 


2.  Fill  out  initial  survey: 

Based  on  your  experience,  complete  this  survey. 


<u 

QJ 

1_ 

CD 

ra 

LA 

J>- 

C 

O 

4—*  1 

LD 


<D 

QJ 

L. 

cn 

CO 

if) 


D 

OJ 

2: 


OJ 
0 J 

CD 

< 


a) 

b) 

d) 


I  have  an  understanding  of  four-bar  linkages. 

I  have  experience  using  computer  simulation  software  (e.g.,  CAD,  FEA,  etc.). 

I  preferto  design  using  software,  rather  than  building  physical  models. 

I  have  experience  building  physical  models. 


0 


Strongly  agree. 


3.  Introduction  to  GIM  software 


4.  Prototype  four-bar  linkage:  [maximum  30  minutes] 

Design  Problem: 

•  Design  a  continuously  rotating  four-bar  linkage  to  draw  the  longest  possible  horizontal  shape. 

•  Goal:  maximize  the  ratio  of  AX /AY 

•  Complete  Likert  Scale  below  and  choose  to  virtually  or  physically  prototype  a  four-bar  linkage. 

•  There  is  no  limit  to  the  number  of  times  you  may  modify  your  design. 

•  Record  your  Starting  and  Completion  Time  below. 


4.  b)  Complete  Likert  Scale: 

Strongly  Disagree. 

Disagree. 

Neutrai. 

Agree. 

Strongly  Agree. 

-2 

-1 

0 

+1 

+2 

a) 

Virtual  prototyping  will  require  less  time  than  building  physical 
prototype(s). 

b> 

Virtual  prototyping  will  be  sufficiently  accurate  to  model  critical 
physics  or  dynamic  motions. 

c) 

Prototyping  a  four-bar  linkage  will  require  many  iterations. 

Use  the  sum  of  your  responses  to  the  above  questions  to 
determine  whether  physical  or  virtual  prototyping  will  be 
pursued  (e.g,,  a  positive  sum  would  suggest  pursuing  virtual 
prototyping). 

Physical 

Virtual 

5.  Fill  out  exit  survey: 

Strongly  disagree. 

Disagree. 

Neutral. 

Agree. 

Strongly  agree. 

-2 

1 

0 

+1 

+2 

a) 

Virtual  prototyping  (vs.  physical  prototyping)  is  the  best  technique  for  designing  four- 
bar  linkages. 

b) 

GIM  software  is  a  useful  tool  for  virtually  prototyping  four-bar  linkages. 

c) 

The  Likert  Scale  above  was  useful  in  choosing  between  virtual  and  physical. 

d) 

1  will  consider  using  virtual  prototyping  in  future  designs. 

e) 

1  chose  the  best  technique  for  my  prototype. 

0  Why  did  you  choose  virtual  or  physical  prototyping? 


6.  Submit  your  physical  prototype, 

or  email  your  virtual  prototype  file  (FirstName_La$tName.gim)  to  [researcher  email  address] 


Appendix  B;  Experimental  Data 
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